Image sensing apparatus having a color interpolation unit and image processing method therefor

ABSTRACT

An image sensing apparatus comprising: a CCD, optical system for forming an image on the CCD; A/D converter for converting an image signal outputted by the CCD to a digital signal; color interpolator for performing color interpolation on the digital signal converted by the A/D converter and generating image data on R, G and B color planes; color space converter for converting RGB color space to YUV colorimetric system; and median filter for reducing pseudo color components, generated by the color interpolator, by controlling color difference signals U and V. By cutting high frequency components of the signal, filtered by the median filter, then thinning out YUV signals and performing JPEG compression, pseudo color components generated by color interpolation can be reduced.

This is divisional of application Ser. No. 09/484,989, filed Jan. 18,2000 now U.S. Pat. No. 6,958,772.

BACKGROUND OF THE INVENTION

The present invention relates to an image sensing apparatus such as adigital camera or digital video camera or the like, and an imageprocessing method for the image sensing apparatus.

A conventional single-plate color digital camera is a lens-integratedsystem, and is generally incapable of exchanging a lens. For thisreason, an optical low pass filter and infrared ray (IR) filter or thelike are provided before a CCD, which generates image signals accordingto light passing through the lens, thereby achieving a certain degree ofeffectiveness in reducing moiré or pseudo colors (erroneous colors).However, in a case of a camera which allows lens exchange, an IR filtercan be provided before a CCD by forming a thin film on a glass surfaceof the CCD. However, securing space for an optical low pass filterbefore a CCD causes to increase the size of the camera body.Furthermore, although such optical low pass filter can reduce moiré orpseudo colors to a certain degree, a problem arises in that spatialfrequency of an image decreases, and as a result, the image losesexact-focused characteristic such as that obtained in silver chloridephotographs. Because of the above reason, the importance of an opticalsystem without an optical low pass filter is increasing.

Furthermore, even if such optical low pass filter is provided, in a caseof a digital camera employing a single-plate CCD, pseudo colors arestill generated when color interpolation is performed. As exemplified bythe Bayer pattern, when a number of R (red) and B (blue) colorcomponents is less than that of G (green) color components, the gapbetween these pixels becomes large, and this causes generation of pseudocolors in color interpolation. Furthermore, a method of generating threecolor planes with a digital filter or the like is available as aconventional color interpolation. However, since the order of thedigital filter is limited, original image data resolution cannot besufficiently expressed.

In view of the above, conventionally proposed is to perform imageprocessing, particularly color interpolation, disclosed in U.S. Pat. No.5,373,322 or U.S. Pat. No. 5,629,734, to obtain high resolution imagedata. This technique is effective to reduce pseudo colors, but noteffective for moiré.

Even with the conventional technique, isolated pseudo colors, generatedparticularly around small characters or the like, cannot completely beeliminated. In view of this, it is proposed to convert color space,e.g., from RGB to L*a*b*, by an application program operating in acomputer, and perform processing such as filtering on a* and b* toremove pseudo colors.

However, even if such conventional technique is employed, pseudo colorsin color interpolation cannot completely be eliminated. This is furtherdescribed below.

As shown in FIG. 19, if vertical white lines are exposed to green (G)and red (R) lines of a CCD at pixel pitch and conventional colorinterpolation is performed, red (R) and yellow (Y) vertical stripes areobtained (in FIG. 19, hatching portion indicates black where data is 0).Similarly, as shown in FIG. 20, if vertical white lines are exposed togreen (G) and blue (B) lines of a CCD at pixel pitch and colorinterpolation is performed, blue (B) and cyan (C) vertical stripes areobtained. Note that even if the stripes in FIGS. 19 and 20 arehorizontal instead of vertical, the same results, i.e., red (R) andyellow (Y) stripes or blue (B) and cyan (C) stripes, are obtained.

Furthermore, as another pattern, if white pixels having a checker flagpattern shown in FIG. 21 are exposed to R and B of a CCD and colorinterpolation is performed, a magenta (M) image is obtained despite theoriginal white color. Still further, if white pixels having a checkerflag pattern shown in FIG. 22 are exposed to G of the CCD and colorinterpolation is performed, a green (G) image is obtained despite theoriginal white color.

In order to remove such pseudo colors, the color space is converted fromRGB to, for instance, L*a*b* and filtering is performed on each of thea* and b* color spaces by an application program operating in a personalcomputer. However, in the case of checker flag patterns shown in FIGS.21 and 22, a low frequency image of green cannot be distinguished fromthat of magenta, making it impossible to determine pseudo colors. If thefrequency band of color difference is forcefully limited for JPEGcompression, a dull (blurred) image is obtained. As a result, althoughthe level of pseudo colors is reduced, the pseudo color components arespread out to the peripheral pixels, causing problems.

Furthermore, if the image data is compressed according to JPEG or thelike without performing above-described pseudo color removingprocessing, block noise or the like may be caused. Moreover, if thefrequency band of color difference is forcefully limited for JPEGcompression, an image becomes blurred. As a result, although the levelof pseudo colors is reduced, the pseudo color components are spread outto the peripheral pixels, causing problems. In order to eliminate suchpseudo colors, it is ideal to employ a three-plate camera using threeimage sensing devices for each color component. However, suchconstruction increases the size and cost of the camera.

SUMMARY OF THE INVENTION

The present invention is made in consideration of the above situation,and has as its object to provide an image sensing apparatus capable ofreducing pseudo colors generated by color interpolation, and an imageprocessing method for the image sensing apparatus.

Furthermore, another object of the present invention is to provide animage sensing apparatus which can reduce pseudo colors withoutincreasing the size or cost of the image sensing apparatus, and an imageprocessing method for the image sensing apparatus.

Furthermore, another object of the present invention is to provide animage sensing apparatus which can reduce pseudo colors with relatively asimple construction, and an image processing method for the imagesensing apparatus.

Furthermore, Another object of the present invention is to provide animage sensing apparatus which can reduce pseudo colors in isolatedpixels with relatively a simple construction, and an image processingmethod for the image sensing apparatus.

In order to attain the above-described objects, the image sensingapparatus according to the present invention has the followingconfiguration.

More specifically, the present invention provides an image sensingapparatus comprising: an image sensing device; image forming means forforming an image on said image sensing device; A/D conversion means forconverting an image signal outputted by said image sensing device into adigital signal; color interpolation means for performing colorinterpolation on the digital signal converted by said A/D conversionmeans and generating image data on a plurality of color planes; colorspace conversion means for converting a color space of the plurality ofcolor planes to a color space of another colorimetric system; andisolated point removing means for reducing a pseudo color, generated bysaid color interpolation means, by controlling a color difference signalconverted by said color space conversion means.

Furthermore, the present invention provides an image sensing apparatuscomprising: an image sensing device; image forming means for forming animage on said image sensing device; A/D conversion means for convertingan image signal outputted by said image sensing device into a digitalsignal; color interpolation means for performing color interpolation onthe digital signal converted by said A/D-conversion means and generatingimage data on a plurality of color planes; pseudo color removing meansfor reducing a pseudo color component included in image data, on whichcolor interpolation is performed by said color interpolation means;color space conversion means for converting a color space of theplurality of color planes, where a pseudo color is reduced by saidpseudo color removing means, to a color space of another colorimetricsystem; and compression means for compressing image data where colorspace is converted by said color space conversion means.

Herein, the isolated point removing means includes an isolated pointremoving filter which replaces a value of a pixel of interest with asubstantial median pixel value of peripheral pixels of the pixel ofinterest. Also, the isolated point removing filter includes a medianvalue filter or median filter.

Moreover, the color interpolation means generates image data in R, G andB planes.

Further, the color space conversion means may convert RGB color space toYUV color space.

Still further, the color space conversion means may convert RGB colorspace to Y, R−Y, B−Y color space.

Still further, the color space conversion means may convert RGB colorspace to G, R−G, B−G color space.

Moreover, the image forming means includes an infrared ray filter, or aninfrared ray filter and optical low pass filter.

Furthermore, the image processing method for the image sensing apparatusaccording to the present invention has the following steps.

More specifically, the present invention provides an image processingmethod for an image sensing apparatus which includes an image sensingdevice and generates an image signal corresponding to an image formed onthe image sensing device, comprising: an A/D conversion step ofconverting an image signal outputted by the image sensing device into adigital signal; a color interpolation step of performing colorinterpolation on the digital signal converted in said A/D conversionstep and generating image data on a plurality of color planes; aseparation step of separating the image data in the plurality of colorplanes into luminance data and color difference data; an extraction stepof extracting a high frequency component from the luminance dataseparated in said separation step; and a pseudo color removing step ofreducing a pseudo color generated in said color interpolation step, inaccordance with the high frequency component of the luminance dataextracted in said extraction step and hue data obtained from the colordifference data.

Herein, the pseudo color removing step comprises: a determination stepof determining whether or not the high frequency component of theluminance data and the hue data fall within a color range of the pseudocolor; and a step of reducing a value of the color difference data,determined to be within the color range in said determination step.

Moreover, in the color space conversion step, it is preferable that thecolor space be converted to YUV, or Y, R−Y, B−Y, or G, R−G, B−G colorspace.

Moreover, the image processing method may further comprise a step ofremoving a high frequency component from a color signal, from which thepseudo color is removed in said pseudo color removing step.

Herein, the color range of the pseudo color includes a color area fromred to yellow, and a color area from blue to cyan.

Other features and advantages of the present invention will be apparentfrom the following description taken in conjunction with theaccompanying drawings, in which like reference characters designate thesame or similar parts throughout the figures thereof.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of the specification, illustrate embodiments of the invention, andtogether with the description, serve to explain the principles of theinvention.

FIG. 1 is a block diagram showing, mainly, a construction of an imageprocessing unit of a digital camera according to the first embodiment ofthe present invention;

FIGS. 2A and 2B are conceptual views explaining pseudo color removalaccording to embodiments of the present invention;

FIG. 3 is a block diagram showing, mainly, a construction of an imageprocessing unit of a digital camera according to the second embodimentof the present invention;

FIG. 4 is a block diagram showing, mainly, a construction of an imageprocessing unit of a digital camera according to the third embodiment ofthe present invention;

FIG. 5 is a block diagram showing, mainly, a construction of an imageprocessing unit of a digital camera according to the fourth embodimentof the present invention;

FIG. 6 is a block diagram showing, mainly, a construction of an imageprocessing unit of a digital camera according to the fifth embodiment ofthe present invention;

FIG. 7 is a block diagram showing a configuration of a pseudo colorremoving processor according to the fifth embodiment;

FIG. 8 is a block diagram showing a configuration after the gammaconverter shown in FIG. 6, for a digital camera according to the sixthembodiment of the present invention;

FIG. 9 is a block diagram showing, mainly, a construction of an imageprocessing unit of a digital camera according to the seventh embodimentof the present invention;

FIG. 10 is a conceptual view for explaining color interpolationaccording to embodiments of the present invention;

FIG. 11 is an explanatory diagram showing a construction of a highfrequency filter according to embodiments of the present invention;

FIG. 12 is a graph explaining processing of a pseudo color suppressionunit according to embodiments of the present invention;

FIG. 13 shows a hue range of pseudo colors in pseudo color suppressionprocessing according to embodiments of the present invention;

FIG. 14 is a flowchart explaining pseudo color removing processingaccording to the seventh and eighth embodiments of the presentinvention;

FIG. 15 is a block diagram showing, mainly, a construction of an imageprocessing unit of a digital camera according to the eighth embodimentof the present invention;

FIG. 16 is a block diagram showing a construction of an image processingunit of a digital camera according to the ninth embodiment of thepresent invention;

FIG. 17 shows a hue range of pseudo colors in pseudo color suppressionprocessing according to the ninth embodiment of the present invention;

FIG. 18 is a block diagram showing a construction of an image processingunit of a digital camera according to the tenth embodiment of thepresent invention;

FIG. 19 is a conceptual view explaining a general pseudo colorgeneration process;

FIG. 20 is a conceptual view explaining a general pseudo colorgeneration process;

FIG. 21 is a conceptual view explaining a general pseudo colorgeneration process; and

FIG. 22 is a conceptual view explaining a general pseudo colorgeneration process.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Preferred embodiments of the present invention will be described indetail in accordance with the accompanying drawings.

First Embodiment

FIG. 1 is a block diagram showing, mainly, a construction of an imageprocessing unit of a digital camera according to a first embodiment ofthe present invention.

Light 1, incident upon the digital camera according to the firstembodiment, passes through lens 2, then the amount of the light isadjusted by a diaphragm 3, and an image sensing device 5 (hereinafterreferred to as a CCD), e.g., CCD or CMOS, is exposed while a shutter(not shown) is open. Before the light 1 is incident upon the CCD 5, anoptical low pass filter 20 limits a spatial frequency of the light 1 toreduce moiré in a way that an optical area of a long wavelength is cutoff by an infrared ray (IR) filter 4 so that the CCD 5 does not detectlight in the infrared region. By the light incident upon the CCD 5, theamount of electric charge corresponding to the intensity of light isaccumulated in the CCD 5. The amount of electric charge is amplified bya predetermined gain by a CDS·AGC 6, and converted to digital data by anA/D converter 7. Image data converted to digital data in the foregoingmanner is transferred to a white balance circuit 8 where R, G and Bgains are adjusted. Then, by a color interpolator 9, the image data isgenerated in, for instance, three color (R, G and B) planes. The imagedata expressed in the R, G and B planes are then transferred to amasking processor 10 where adjustment is made for the hue of the R, Gand B colors. Then, a gamma converter 11 performs necessary processingfor displaying an image on a display or the like.

Next, compression according to JPEG or the like is performed on theimage data since the data in the R, G and B color planes has a largeamount of data.

Herein, first, an RGB/YUV converter 12 converts the image data from R, Gand B signals to Y color difference signals. For instance, R, G and Bdata are respectively converted to Y, U and V as follows:Y=0.29900×R+0.58700×G+0.11400×B  (1)U=−0.16874×R−0.33126×G+0.50000×B  (2)V=0.50000×R−0.41869×G−0.08131×B  (3)

Next, with respect to U and V, each representing a color differencesignal, a pseudo color generated by color interpolation is reduced bymedian filters 13 a and 13 b. Low pass filters (LPF) 14 a and 14 bremove high frequency components to limit the band. Then, a thinningcircuit 15 performs thinning processing which complies with apredetermined compression method, e.g., from Y:U:V=4:4:4 to Y:U:V=4:2:2or Y:U:V=4:1:1. Then, a compression circuit 16 adopting JPEG or the likeperforms compression.

FIGS. 2A and 2B explain the principle of processing performed by themedian filters 13 a and 13 b according to the first (and subsequent)embodiment. Herein, FIG. 2A shows 3×3 pixels peripheral to a pixel ofinterest, and FIG. 2B shows a graph of pixel values arranged in theascending order.

Herein, for instance, 3×3 pixels peripheral to a pixel of interest a*22are extracted, and the pixel values (color difference values) arearranged in the ascending order. Then, the value “20” of the pixel ofinterest is replaced with the value “10” of a*33, which is thesubstantial median value of the nine pixels. Such processing isperformed by the median filters 13 a and 13 b, serving as isolated pointremoving filters. By this, isolated pseudo colors can be reduced withoutlargely decreasing resolution. Note that since the configuration of themedian filter is well known (e.g., disclosed in Japanese PatentApplication Laid-Open No. 5-233804 or No. 5-12437), detailed descriptionthereof will be omitted.

As has been described above, according to the first embodiment, a colorspace of an inputted image signal is converted from RGB to YUV colorspace, and an isolated pixel is removed from color difference signals Uand V of the YUV signal by the median filter or the like. By virtue ofthis, pseudo colors generated by color interpolation can be reduced.

Second Embodiment

In the above-described first embodiment, color conversion from RGB toYUV color space is performed and pseudo color reduction processing isperformed in the YUV color space before image compression is performedaccording to JPEG or the like. However, pseudo color reductionprocessing is not necessarily performed in YUV color space, but may beperformed in another color space.

According to the second embodiment of the present invention, pseudocolor removal is performed by a median filter, after a color space ofinput image data is converted from RGB color space to Y, R−Y, B−Y colorspace.

FIG. 3 is a block diagram showing a construction of an image processingunit of a digital camera according to the second embodiment. For thecomponents that are common to the construction shown in FIG. 1, the samereference numerals are assigned and description thereof will be omitted.The second embodiment largely differs from the foregoing firstembodiment in that the second embodiment does not necessitate JPEGcompression and that the YUV conversion can be eliminated since thepseudo-color-removed image data is stored in a memory 19. Therefore, theconstruction of the image processing unit can be made simple, whichcontributes to the effect of the second embodiment. This is alsoapplicable to a case of performing compression according to JPEG or thelike.

According to the second embodiment, in order to achieve reduction in theamount of calculation compared to the case of converting data from RGBto YUV color space, image data is converted from RGB color space to Y,R−Y, B−Y color space by, for instance, using a Y, R−Y, B−Y converter 17a.

Herein, for instance, color space conversion is executed in thefollowing manner:Y=0.29900×R+0.58700×G+0.11400×B   (4)R−Y=R−Y   (5)B−Y=B−Y   (6)

Then, the median filters 17 b and 17 c respectively executes medianfiltering of the R−Y signal and B−Y signal. Herein, the equation can befurther simplified as follows to obtain luminance signal Y, colordifference signals R−Y and B−Y:Y=R+2×G+B=Ye  (7)R−Y=R−Ye  (8)B−Y=B−Ye  (9)

In this case, R−Ye and B−Ye are calculated with a luminance signal Yewhich is a simplified (approximate) form of the luminance signal Y. Byexecuting median filtering respectively to R−Ye and B−Ye, pseudo colorscan be removed. Although the resultant values are slightly differentfrom values of Y, R−Y and B−Y, a similar effect is achieved andcalculation can be performed easily. Next, an RGB converter 17 drespectively converts the Y, R−Y and B−Y signals back to R, G and Bsignals. The image data represented by R, G and B signals is subjectedto masking by a masking processor 10 and to a gamma conversion by agamma converter 11, and then stored in the non-volatile memory 19.

Third Embodiment

In the above-described second embodiment, the luminance signal Y isconsciously calculated from R, G and B signals. However, in imageprocessing, G signal of the R, G and B may substitute for the luminancesignal Y.

The third embodiment is characterized by employing G, R−G and B−Gsignals instead of Y, R−Y and B−Y signals employed in the secondembodiment, and executing median filtering to R−G and B−G signalsrespectively.

FIG. 4 is a block diagram showing a construction of an image processingunit of a digital camera according to the third embodiment of thepresent invention. For the components that are common to theconstruction shown in FIG. 3, the same reference numerals are assignedand description thereof will be omitted.

According to the third embodiment, the amount of calculation is reducedfurther when compared against the case of converting R, G and B signalsto luminance signal Y and color difference signals R−Y and B−Y. The G,R−G, B−G converter 18 a generates G, R−G and B−G signals from R, G and Bsignals using the following equations:Y=G   (10)R−Y=R−G  (11)B−Y=B−G  (12)

The median filters 18 b and 18 c respectively execute filtering to theR−G and B−G signals, thereby removing pseudo colors generated by colorinterpolation. Then, the RGB converter 18 d converts the G, R−G and B−Gsignals back to RGB signals, which are then subjected to the maskingprocessor 10 and gamma converter 11, and then stored in the non-volatilememory 19.

As has been described above, since the third embodiment employs G signalin place of luminance signal Y, pseudo color reduction can be realizedwith a less amount of calculation, although there is a possibility thatluminance components of the RGB signals may be influenced by the resultof median filtering.

Fourth Embodiment

The above-described first to third embodiments describe the case ofemploying an optical low pass filter before the CCD 5. Note that theconfiguration before the CCD 5 is not shown in FIGS. 3 and 4.

The fourth embodiment describes a case where no problem arises eventhough the optical system does not comprise such optical low passfilter.

FIG. 5 is a block diagram showing a construction of an image processingunit of a digital camera according to the fourth embodiment of thepresent invention. For the components that are common to theconstruction shown in FIG. 1, the same reference numerals are assignedand description thereof will be omitted. The fourth embodiment ischaracterized in that the IR filter 4 for cutting infrared rays isprovided before the CCD 5, and that the optical low pass filter 20employed in FIG. 1 is removed. The configuration after the CCD 5 is thesame as that shown in block diagram of FIG. 1.

However, when the optical low pass filter 20 is removed, pseudo colorsare generated more than the case of including the optical low passfilter 20. In view of this, the number of elements in median filters 130a and 130 b in the fourth embodiment is increased from the number ofelements “9” of the median filters according to first to thirdembodiments. By this, the median filtering can be executed in a widerarea, and pseudo colors can be reduced.

Accordingly, since the fourth embodiment is realized by simply mountingthe IR filter 4 on the glass surface of the CCD 5, the apparatus mainbody can be made small and cost reduction can be achieved. In addition,image signals of high resolution can be obtained.

For the isolated point removing filter, filtering called smoothing (EdgePreserving Smoothing), which preserves edges, may be employed besidesthe above-described median filter. More specifically, a window W(i, j)is set with a pixel f(x, y) as its center, and an output g(x, y) iscalculated. Assuming that the window size is m×n, the following equationstands:

${g\left( {x,y} \right)} = {\sum\limits_{i = 1}^{m}{\sum\limits_{j = 1}^{n}{{w\left( {i,j} \right)} \times {f\left( {{x - i},{y - i}} \right)}}}}$For instance, assuming a matrix M in which data “1” is arranged in 3×3matrix, the following equation stands:W=M/9

For an alternative isolated point removing filter, another smoothing(Maximum Homogeneity Smoothing) which preserves edges may be employed.This smoothing enables selection of a most uniform area in theneighborhood of a pixel. When the weight coefficient Fi is defined asfollows:weight coefficient Fi={min(σi)}/σiwhere an average value of each block is μi, distribution is σi, and m(arbitrary),the output g(x, y) is obtained in the following manner:

$\left. {{g\left( {x,y} \right)} = {\sum\limits_{i = 1}^{8}{\left\{ {Fi}^{m} \right){\mu\mathbb{i}}}}} \right\}/{\sum\limits_{i = 1}^{8}{Fi}^{m}}$

Besides the above, filtering called hysteresis smoothing may beemployed. More specifically, a hysteresis characteristic is produced fora data value, and noise is absorbed by the hysteresis characteristic.Any of the aforementioned filters may be used as long as interpolationor smoothing is performed by using outputs of peripheral pixels of anisolated point having a pixel value largely different from peripheralpixel values.

Fifth Embodiment

FIG. 6 is a block diagram showing, mainly, a construction of an imageprocessing unit of a digital camera according to the fifth embodiment ofthe present invention. For the components that are common to theforegoing embodiments, the same reference numerals are assigned.

Light 1, incident upon the digital camera according to the fifthembodiment, passes through the lens 2, then the amount of the light isadjusted by the diaphragm 3, and the image sensing device 5 (hereinafterreferred to as a CCD), e.g., CCD or CMOS, is exposed while a shutter(not shown) is open. Before the light 1 is incident upon the CCD 5, afrequency region of a long wavelength is cut off by the IR filter 4 sothat the CCD 5 does not detect light in the infrared region. By thelight incident upon the CCD 5, the amount of electric chargecorresponding to the intensity of light is accumulated in the CCD 5. Theamount of electric charge is amplified by a predetermined gain by theCDS·AGC 6, and converted to digital data by the A/D converter 7. Imagedata converted to digital data in the foregoing manner is transferred tothe white balance circuit 8 where R, G and B gains are adjusted. Then,by the color interpolator 9, the image data is generated in, forinstance, three color (R, G and B) planes. The image data expressed inthe R, G and B planes are then transferred to the masking processor 10where adjustment is made for the hue of the R, G and B colors. Then, thegamma converter 11 performs necessary processing for displaying an imageon a display or the like.

Next, a pseudo color removing processor 21 reduces pseudo colorsgenerated by color interpolation of the color interpolator 9. Further,compression processing according to JPEG or the like is performed on theimage data since the data in the R, G and B color planes has a largeamount of data. Herein, first, the RGB/YUV converter 12 converts theimage data from R, G and B signals to Y color difference signal, andwith respect to each of the color difference signals U and V, the LPF 14a and 14 b respectively perform low pass filtering to limit the band.Then, the thinning circuit 15 performs thinning processing whichcomplies with a predetermined compression method, e.g., from Y:U:V=4:4:4to Y:U:V=4:2:2 or Y:U:V=4:1:1. Then, a JPEG compression circuit 16performs compression according to JPEG.

FIG. 7 is a block diagram showing a configuration of the pseudo colorremoving processor 21 according to the fifth embodiment.

Assume that image data is generated in R, G and B planes by the colorinterpolator 9. The RGB/XYZ converter 21 a converts image data from RGBcolor space to XYZ color space. The conversion is executed by thefollowing equations (13) to (15):X=2.7689×R+1.7517×G+1.1302×B  (13)Y=1.0000×R+4.5907×G+0.0601×B  (14)Z=0.0000×R+0.0565×G+5.5943×B  (15)

Then, XYZ/L*a*b* converter 21 b converts the image data from XYZ colorspace to L*a*b* color space. The conversion is executed by the followingequations (16) to (18):L*=116(Y/Y ₀)^(1/3)−16(Y/Y ₀>0.008856)   (16)a*=500[(X/X ₀)^(1/3)−(Y−Y ₀)^(1/3)]  (17)b*=200[(Y/Y ₀)^(1/3)−(Z−Z ₀)^(1/3)]  (18)

X₀, Y₀ and Z₀ represent tristimulus values of the perfect diffusesurface.

Herein, median filters 21 c and 21 d, each serving as an isolated pointfilter, execute median filtering (median value filtering) respectivelyto a* and b* of the L*a*b*.

Since the principle of the processing performed by the median filters 21c and 21 d is the same as the aforementioned description shown in FIGS.2A and 2B, description thereof will not be provided herein.

Then, L*a*b*/XYZ converter 21 e converts image data from L*a*b* colorspace to XYZ color space. Finally, XYZ/RGB converter 21 f converts theimage data from XYZ color space to RGB color space. By this, medianfiltering can be executed only in a color space that gives smallinfluence to the luminance (resolution). Therefore, the decrease inresolution of an image can be prevented.

Sixth Embodiment

FIG. 8 is a block diagram showing a configuration according to the sixthembodiment of the present invention.

In the foregoing fifth embodiment, image data in RGB color space, whichhas been outputted by the pseudo color removing processor 21, isconverted from RGB color space to YUV color space by the RGB/YUVconverter 12 to be subjected to JPEG compression. This YUV conversion isexecuted by the aforementioned equations (1) to (3).

According to the sixth embodiment, instead of the pseudo color removingprocessor 21, the median filters (median value filters) 17 b and 17 cexecute filtering to U and V signals outputted by the RGB/YUV converter12. Then, to perform JPEG compression, the LPF 14 a and 14 b perform lowpass filtering to color difference data to limit the band of the colordifference signals. Then, the thinning circuit 15 performs thinning intoa predetermined format, e.g., from Y:U:V=4:4:4 to Y:U:V=4:2:2 orY:U:V=4:1:1, and compression is performed by the JPEG compressioncircuit 16.

For the isolated point removing filter, filtering called smoothing (EdgePreserving Smoothing) for preserving edges, which has been described inthe fourth embodiment, may be employed besides the above-describedmedian filter.

Furthermore, for the isolated point removing filter, smoothing (MaximumHomogeneity Smoothing) for preserving edges, which has been described inthe fourth embodiment, may be employed.

Besides the above, filtering called hysteresis smoothing may beemployed. More specifically, a hysteresis characteristic is produced fora data value, and noise is absorbed by the hysteresis characteristic.Any of the aforementioned filters may be used as long as interpolationor smoothing is performed by using outputs of peripheral pixels of anisolated point having a pixel value largely different from peripheralpixel values.

Seventh Embodiment

A characteristic-feature of the seventh embodiment is described beforeexplaining a specific example of the seventh embodiment. To eliminatethe above-described pseudo colors, it is ideal to employ three-platecamera where CCDs are provided for each of the R, G and B colorcomponents. With such structure, the size of a camera increases. In theseventh embodiment, an attention is directed to the fact that pseudocolors generated from red (R) to yellow (Y) and from blue (B) to cyan(C) have high frequencies. Thus, the characteristic feature of theseventh embodiment is to suppress generation of pseudo colors byreducing the gain of a color difference signal when the luminance signalhas a high frequency component and the hue falls within a predeterminedrange.

FIG. 9 is a block diagram showing, mainly, a construction of an imageprocessing unit of a digital camera according to the seventh embodimentof the present invention.

Light 1, incident upon the camera, passes through the lens 2, then theamount of the light is adjusted by the diaphragm 3, and the imagesensing device 5 (hereinafter referred to as a CCD), e.g., CCD or CMOS,is exposed while a shutter (not shown) is open. Before the light 1 isincident upon the CCD 5, a frequency area (band) of a long wavelength iscut off by the IR filter 4 so that the CCD 5 does not detect light inthe infrared region. The light incident upon the CCD 5 is accumulated inthe CCD 5 as an electric charge, and outputted as an electric signal.The electric signal is amplified by a predetermined gain by the CDS·AGC6, and converted to digital data by the A/D converter 7. Image dataconverted to digital data in the foregoing manner is transferred to thewhite balance circuit 8 where R, G and B gains are adjusted. Then, inthe color interpolator 9, the image data is generated in, for instance,three color (R, G and B) planes as shown in FIG. 10. The image dataexpressed in the R, G and B planes are then transferred to the maskingprocessor 10 where adjustment is made for the hue of the R, G and Bcolors. Then, the gamma converter 11 performs necessary processing fordisplaying an image on a display or the like. Further, compressionprocessing according to JPEG or the like is performed on the image datasince the data in the R, G and B color planes has a large amount ofdata.

First, RGB/Y converter 22 converts RGB color space to color space of Ycolor difference signal. The conversion is executed by, for instance,above-mentioned equations (1) to (3).

To detect high frequency components and hue, the image data, expressedin the R, G and B planes by the color interpolator 9, is subjected tocalculation of a luminance signal Y by the RGB/Y converter 22, using theaforementioned equation (1). Next, to obtain a high frequency componentof the obtained luminance signal Y, a spatial high pass filter (HPF) 23executes spatial high pass filter computation as shown in FIG. 11. Theluminance signal Y, obtained by the RGB/Y converter 22, is transferredto a hue detection circuit 24 for detecting a hue, where colordifference signals R−Y and B−Y are calculated and a hue is detectedbased on the calculated results. The detected hue data is outputted to apseudo color suppression circuit 13. Referring to FIG. 11, spatial highpass filter computation is executed by multiplying an input signal by amatrix 300, and subtracting the input signal from the multiplicationresult by a subtractor 301.

The pseudo color suppression circuit 13 suppresses pseudo colors byreducing gains of U and V, outputted by the RGB/Y converter 22, based onthe high frequency component of the luminance signal Y obtained by thespatial high pass filter 23 and the hue data obtained by the huedetection circuit 24.

The pseudo color suppression processing is described with reference toFIGS. 12 and 13.

First, it is determined whether or not a color of the pixel of interestfalls within a predetermined range (area indicated by hatching in FIG.13) in Y, R−Y, B−Y color space. If the color is determined to be in therange where it is considered as a pseudo color, gains of the colordifference signals U and V, having a predetermine value of 1 or largeras shown in FIG. 12, are reduced in accordance with a high frequencycomponent value of the luminance signal outputted by the spatial highpass filter 23. By this, pseudo colors can be suppressed.

Pseudo color suppression processing performed in the above-describedconfiguration is explained. A high frequency component, such as thathaving a vertical stripe pattern or a checker flag pattern describedwith reference to FIGS. 19 to 22, is extracted by the spatial HPF 23.The aforementioned pseudo color generation is prevented by reducinggains of color difference signals U and V in the extracted highfrequency area, which may cause pseudo colors described in FIGS. 19 to22 (for instance, predetermined areas of R and Y components in FIG. 19,predetermined areas of C and B components in FIG. 20, or predeterminedareas of R and B components in FIGS. 21 and 22, all of which correspondto the hatching portion in FIG. 13).

Next, the low pass filter (LPF) 14 execute low pass filtering in thehorizontal or vertical direction. Then, the thinning circuit 15 performsthinning processing of the color difference signals, in accordance witha predetermined JPEG format, e.g., from Y:U:V=4:4:4 to Y:U:V=4:2:2 orY:U:V=4:1:1. Then, the JPEG compression circuit 16 performs JPEGcompression. By this, a compressed image having less pseudo colors canbe generated.

Note herein that instead of calculating the luminance signal Y by theaforementioned equation (1), the calculation may be executed with asimplified luminance signal Ye. For instance, aforementioned equation(7), Ye=R+2×G+B, may be used.

Alternatively, a green (G) signal component may be used as a luminancesignal without calculating the luminance signal.

FIG. 14 is a flowchart explaining pseudo color removing processingaccording to the seventh embodiment of the present invention.

Referring to FIG. 14, in step S1, R, G and B signals of an image, uponwhich color interpolation processing has been performed, are inputtedand a luminance signal Y is generated from the R, G and B signals by theRGB/Y converter 22. In step S2, high frequency components of theluminance signal Y are extracted by the spatial high pass filter (HPF)23. In step S3, differences are respectively calculated between each ofR, B signals and the luminance signal Y by the hue detection circuit 24,and color difference signals R−Y and B−Y are generated. In step S4, ahue signal is generated based on the color difference signals, and it isdetermined whether or not the hue signal falls within a color range thatis considered as pseudo colors. If it is determined as a pseudo color instep S5, the control proceeds to step S6 where gains of the colordifference signals in the pseudo color portion are reduced in accordancewith the amount of high frequency component of the pixel. By this, forinstance, in the color space corresponding to the hatching area in FIG.13, pseudo color components generated by color interpolation can besuppressed. Then the process proceeds to step S7, isolated pixels of theimage are removed.

As described above, the seventh embodiment achieves an effect ofpreventing pseudo colors generation from red to yellow and from blue tocyan.

Eighth Embodiment

A characteristic feature of the eighth embodiment is described beforeexplaining a specific example of the eighth embodiment. To eliminate theabove-described pseudo colors, it is ideal to employ three-plate camerawhere CCDs are provided for each of the R, G and B color components.With such structure, the size of a camera increases. In the eighthembodiment, an attention is directed to the fact that pseudo colorsgenerated from red (R) to yellow (Y) and from blue (B) to cyan (C) havehigh frequencies. Thus, the characteristic feature of the eighthembodiment is to suppress generation of pseudo colors by removing anisolated point of a luminance signal, having a high frequency componentand, if the hue falls within a predetermined range, reducing the gain ofthe color difference signal.

FIG. 15 is a block diagram showing, mainly, a construction of an imageprocessing unit of a digital camera according to the eighth embodimentof the present invention.

The pseudo color suppression circuit 13 suppresses pseudo colors byreducing gains of U and V, outputted by the RGB/YUV converter 22, basedon a high frequency component of the luminance signal Y obtained by thespatial high pass filter 23 and hue data obtained by the hue detectioncircuit 24. The pseudo color suppression processing is performed in thesame manner as that of the above-described seventh embodiment.

Among the above-described pseudo colors, patterns of pseudo colors fromred (R) to yellow (Y) and from blue (B) to cyan (C) can be determined aspseudo colors by detecting high frequency components of the luminancesignal because the degree of luminance of pseudo colors are differentfrom others. However, pseudo colors generated around a character areoften isolated, and the degree of luminance of ht pseudo colors are notalways different from the peripheral pixel luminance.

In view of the above, according to the eighth embodiment, the medianfilters 24 a and 24 b remove isolated points with respect to the colordifference signals U and V and further perform pseudo color removal.FIGS. 2A and 2B explain this processing. Herein, elements having 3×3pixel area including a pixel of interest are arranged in the ascendingorder, and the value of the pixel of interest is replaced with a medianvalue of the arranged pixels. By this, a pixel, having a color ofextremely different hue from the hues of peripheral pixels, can beeliminated. Particularly, since the low pass filtering is not executedin the eighth embodiment, resolution does not decrease largely, thusdoes not influence peripheral pixels.

Pseudo color suppression processing performed in the above-describedconfiguration is explained. A high frequency component, such as thathaving a vertical stripe pattern or a checker flag pattern describedwith reference to FIGS. 19 to 22, is extracted by the spatial HPF 23.The aforementioned pseudo color generation is prevented by reducinggains of color difference signals U and V in the extracted highfrequency area, which may cause the pseudo colors described in FIGS. 19to 22 (for instance, predetermined areas of R and Y components in FIG.19, predetermined areas of C and B components in FIG. 20, orpredetermined areas of R and B components in FIGS. 21 and 22, all ofwhich correspond to the hatching portion in FIG. 13). After pseudocolors are suppressed in the foregoing manner, the median filters 24 aand 24 b execute above-described filtering, thereby removing isolatedpixels.

Next, the low pass filter (LPF) 14 a and 14 b execute low pass filteringin the horizontal or vertical direction. Then, the thinning circuit 15performs thinning processing of the color difference signals, inaccordance with a predetermined JPEG format, e.g., from Y:U:V=4:4:4 toY:U:V=4:2:2 or Y:U:V=4:1:1. Then, the JPEG compression circuit 16performs JPEG compression. By this, a compressed image having lesspseudo colors can be generated.

Note that, to calculate the luminance signal Y, the aforementionedequation (7) may be used.

Alternatively, a green (G) signal component may be used as a luminancesignal without calculating the luminance signal.

Since the pseudo color removal processing according to the eighthembodiment is performed in the same manner as that explained in theflowchart in FIG. 14, detailed description will be omitted herein.

Ninth Embodiment

In the foregoing seventh and eighth embodiments, means for detectingpseudo colors is provided after the color interpolator 9 and pseudocolor detection is performed using signals in the R, G and B planes. Thereason is that, since processing such as masking processing, γconversion and so forth that may cause changes in hue are performedafter the color interpolator 9, the hue range of the detected pseudocolors may change. Because of this reason, the luminance signal Y iscalculated to obtain color difference signals R−Y and B−Y. However, thepseudo color detection means does not have to be provided in thisposition. For instance, in order to perform JPEG compression, RGB colorspace must be converted to a color space of luminance or colordifference. Therefore, pseudo colors can be detected from these signals.The ninth embodiment relates to such case. Note that, as mentionedabove, there is a possibility that the hue range of pseudo colorschanges depending on images.

FIG. 16 is a block diagram showing a construction of an image processingunit of a digital camera according to the ninth embodiment of thepresent invention. For the components that are common to theconstruction shown in FIG. 9, the same reference numerals are assigned.

Light 1, incident upon the camera, passes through the lens 2, then theamount of the light is adjusted by the diaphragm 3, and the CCD 5 isexposed while a shutter (not shown) is open. Before the light 1 isincident upon the CCD 5, a frequency area of a long wavelength is cutoff by the IR filter 4 so that the CCD 5 does not detect light in theinfrared region. The light incident upon the CCD 5 is accumulated in theCCD 5 as an electric charge, and outputted as an electric signal. Theelectric signal is amplified by a predetermined gain by the CDS·AGC 6,and converted to digital data by the A/D converter 7. Image dataconverted to digital data in the foregoing manner is transferred to thewhite balance circuit 8 where R, G and B gains are adjusted. Then, inthe color interpolator 9, the image data is generated in, for instance,three color (R, G and B) planes as shown in FIG. 2. The image dataexpressed in the R, G and B planes are then transferred to the maskingprocessor 10 where adjustment is made for the hue of the R, G and Bcolors. Then, the gamma converter 11 performs necessary processing fordisplaying an image on a display or the like. Then, RGB/YUV converter 12converts the image data from RGB color space to color space of Y colordifference signal.

Among the converted signals, luminance signal Y is transmitted to thespatial high pass filter (HPF) 25 for detecting high frequencycomponents of the signal by the filtering shown in FIG. 11. Then, a huedetection circuit 26 determines whether or not a pixel of interest fallswithin the hue range of pseudo colors. Herein, instead of theabove-described color space R−Y and B−Y shown in FIG. 13, U and Voutputted by the RGB/YUV converter 12 are used as shown in FIG. 17.

As similar to the seventh and eighth embodiments, if it is determinedthat a color of the pixel of interest falls within the hue range ofpseudo colors, the pseudo color suppression circuit 13 reduces gains ofthe color difference signals U and V, having a value of 1 or larger asshown in FIG. 12, in accordance with an output value of the spatial HPF25.

In this case, for instance, it may be necessary to change, for instance,the hue range depending on the white balance. For instance, when thegain of red is increased, the hue range is shifted towards red, or a hueangle is widened.

Next, the LPF 14 a and 14 b execute low pass filtering in the horizontalor vertical direction. Then, the thinning circuit 15 performs thinningprocessing of the color difference signals, in accordance with apredetermined JPEG format, e.g., from Y:U:V=4:4:4 to Y:U:V=4:2:2 orY:U:V 4:1:1. Then, the JPEG compression circuit 16 performs JPEGcompression.

By this, a compressed image having less pseudo colors can be generated.

Tenth Embodiment

In the foregoing seventh and eighth embodiments, means for detectingpseudo colors is provided after the color interpolator 9 and pseudocolor detection is performed using signals in the R, G and B planes. Thereason is that, since processing such as masking processing, γconversion and so forth that may cause changes in hue are performedafter the color interpolator 9, the hue range of the detected pseudocolors may change. Because of this reason, the luminance signal Y iscalculated to obtain color difference signals R−Y and B−Y. However, thepseudo color detection means does not have to be provided in thisposition. For instance, in order to perform JPEG compression, RGB colorspace must be converted to a color space of luminance and colordifference. Therefore, pseudo colors can be detected from these signals.With reference to the foregoing eighth embodiment (FIG. 15), in themedian filters 24 a and 24 b where pseudo color suppression isperformed, delay is generated between the luminance signal Y and colordifference signals U and V. Because of this, the luminance signal mustalso be delayed. The tenth embodiment makes use of this delay.

FIG. 18 is a block diagram showing a construction of an image processingunit of a digital camera according to the tenth embodiment of thepresent invention. For the components that are common to theconstruction shown in FIG. 15, the same reference numerals are assigned.

Light 1, incident upon the camera, passes through the lens 2, then theamount of the light is adjusted by the diaphragm 3, and the CCD 5 isexposed while a shutter (not shown) is open. Before the light 1 isincident upon the CCD 5, a frequency area of a long wavelength is cutoff by the IR filter 4 so that the CCD 5 does not detect light in theinfrared region. The light incident upon the CCD 5 is accumulated in theCCD 5 as an electric charge, and outputted as an electric signal. Theelectric signal is amplified by a predetermined gain by the CDS·AGC 6,and converted to digital data by the A/D converter 7. Image dataconverted to digital data in the foregoing manner is transferred to thewhite balance circuit 8 where R, G and B gains are adjusted. Then, inthe color interpolator 9, the image data is generated in, for instance,three color (R, G and B) planes as shown in FIG. 2. The image dataexpressed in the R, G and B planes are then transferred to the maskingprocessor 10 where adjustment is made for the hue of the R, G and Bcolors. Then, the gamma converter 11 performs necessary processing fordisplaying an image on a display or the like. Then, RGB/YUV converter 12converts the image data from RGB color space to color space of Y colordifference signal.

Among the converted signals, luminance signal Y is transmitted to thespatial high pass filter (HPF) 25 for detecting high frequencycomponents of the luminance signal by the filtering shown in FIG. 11.Meanwhile, color difference signals U and V are transmitted to themedian filters 24 a and 24 b for executing the above-described medianfiltering shown in FIGS. 2A and 2B. By virtue of this, while highfrequency components of the luminance signal Y are detected, medianfiltering is executed to the color difference signals. Therefore, it ispossible to reduce wasteful use of a memory caused by delay of one ofthe signals.

Next, the hue detection circuit 26 determines whether or not a pixel ofinterest falls within the hue range of pseudo colors. Herein, instead ofthe above-described color space R−Y and B−Y shown in FIG. 13, U and Voutputted by the RGB/YUV converter 12 are used as shown in FIG. 17.

As similar to the eighth embodiment, if it is determined that a color ofthe pixel of interest falls within the hue range of pseudo colors, thepseudo color suppression circuit 13 reduces gains of the colordifference signals U and V, having a value of 1 or larger as shown inFIG. 12, in accordance with an output value of the spatial HPF 25.

In this case, for instance, it may be necessary to change, for instance,the hue range depending on the white balance. For instance, when thegain of red is increased, the hue range is shifted to the red, or a hueangle is widened.

Next, the LPF 14 a and 14 b execute low pass-filtering in the horizontalor vertical direction. Then, the thinning circuit 15 performs thinningprocessing of the color difference signals, in accordance with apredetermined JPEG format, e.g., from Y:U:V=4:4:4 to Y:U:V=4:2:2 orY:U:V=4:1:1. Then, the JPEG compression circuit 16 performs JPEGcompression.

By this, a compressed image having less pseudo colors can be generated.

The present invention can be applied to a system constituted by aplurality of devices (e.g., host computer, interface, reader, printer)or to an apparatus comprising a single device (e.g., copying machine,facsimile machine).

Further, the object of the present invention can also be achieved byproviding a storage medium storing program codes for performing theaforesaid processes to a computer system or apparatus (e.g., a personalcomputer), reading the program codes, by a computer (CPU or MPU) of thesystem or apparatus, from the storage medium, then executing theprogram.

In this case, the program codes read from the storage medium realize thefunctions according to the embodiments, and the storage medium storingthe program codes constitutes the invention.

Further, the storage medium, such as a floppy disk, a hard disk, anoptical disk, a magneto-optical disk, CD-ROM, CD-R, a magnetic tape, anon-volatile type memory card, and ROM can be used for providing theprogram codes.

Furthermore, besides aforesaid functions according to the aboveembodiments are realized by executing the program codes which are readby a computer, the present invention includes a case where an OS(operating system) or the like working on the computer performs a partor the entire processes in accordance with designations of the programcodes and realizes functions according to the above embodiments.

Furthermore, the present invention also includes a case where, after theprogram codes read from the storage medium are written in a functionexpansion card which is inserted into the computer or in a memoryprovided in a function expansion unit which is connected to thecomputer, CPU or the like contained in the function expansion card orunit performs a part or the entire process in accordance withdesignations of the program codes and realizes functions of the aboveembodiments.

As has been set forth above, according to the above-describedembodiments, RGB color space is converted to YUV or Y, R−Y, B−Y or G,R−G, B−G color space, and isolated point removing filtering (e.g.,median filter or the like) is executed respectively to color differencesignals in UV, or R−Y, B−Y or R−G, B−G color space, thereby enablingreduction of pseudo colors generated by color interpolation.

Furthermore, according to the above-describe embodiments, even if anoptical low pass filter is not provided before the image sensing device,such as CCD or the like, pseudo color components can be reduced byconverting RGB color space to L*a*b* or YUV color space, then performingfiltering to color difference signals such as a* and b* signals or U andV signals with the isolated point removing filter, and then performingcompression according to JPEG or the like. Accordingly, an image, havingless block distortion and reduced pseudo color components, can beobtained.

Furthermore, according to the above-described embodiments, pseudo colorsgenerated by color interpolation can be suppressed by detecting highfrequency components of a luminance signal and pseudo colors generatedfrom red to yellow and from blue to cyan in the hue, and then reducinggains of color difference signals U and V.

Still further, according to the above-described embodiments, pseudocolors generated by color interpolation can be suppressed by detectinghigh frequency components of a luminance signal and pseudo colorsgenerated from red to yellow and from blue to cyan in the hue, thenreducing gains of color difference signals U and V, and furthereliminating isolated pixels.

Still further, according to the above-described embodiments, pseudocolors generated by color interpolation can be suppressed by executingmedian filtering to color difference signals in the color space ofluminance and color difference signals, then detecting high frequencycomponents of a luminance signal and pseudo colors generated from red toyellow and from blue to cyan in color difference signals, and reducinggains of color difference signals U and V.

Still further, according to the above-described embodiments, byexecuting median filtering to color difference signals while detectinghigh frequency components of a luminance signal, it is possible toeliminate a processing circuit necessary to match delayed timing of theluminance signal and color difference signals.

The present invention is not limited to the above embodiments andvarious changes and modifications can be made within the spirit andscope of the present invention. Therefore, to apprise the public of thescope of the present invention, the following claims are made.

1. An image sensing apparatus comprising: an image sensing device; afocusing unit configured to focus an image on said image on said imagesensing device; an A/D converter configured to convert an image signaloutputted by said image sensing device into a digital signal; a colorinterpolation unit configured to perform color interpolation on thedigital signal converted by said A/D converter and generate image dataon a plurality of color planes; a separation unit configured to separatethe image data in the plurality of color planes into luminance data andcolor difference data; an extraction unit configured to extract a highfrequency component from the luminance data separated by said separationunit; a pseudo color removing unit configured to reduce a colorcomponent generated by said color interpolation unit, in accordance withthe high frequency component of the luminance data extracted by saidextraction unit and hue data obtained from the color difference data; adetermination unit configured to determine whether or not the highfrequency component of the luminance data and the hue data fall within acolor range of a pseudo color; and a reduction unit configured to reducea value of the color difference data, determined to be within the colorrange by said determination unit.
 2. The image sensing apparatusaccording to claim 1, further comprising a low pass filter for removinga high frequency component from the color difference data whose value isreduced by said reduction unit.
 3. The image sensing apparatus accordingto claim 1, wherein the color range of the pseudo color includes a colorarea from red to yellow, and a color area from blue to cyan.
 4. An imageprocessing method for an image sensing apparatus which includes an imagesensing device and generates an image signal corresponding to an imageformed on the image sensing device, comprising: an A/D conversion stepof converting an image signal outputted by the image sensing device intoa digital signal; a color interpolation step of performing colorinterpolation on the digital signal converted in said A/D conversionstep and generating image data on a plurality of color planes; aseparation step of separating the image data in the plurality of colorplanes into luminance data and color difference data; an extraction stepof extracting a high frequency component from the luminance dataseparated in said separation step; a pseudo color removing step ofreducing a color component generated in said color interpolation step,in accordance with the high frequency component of the luminance dataextracted in said extraction step and hue data obtained from the colordifference data; a determination step of determining whether or not thehigh frequency component of the luminance data and the hue data fallwithin a color range of a pseudo color; and a reduction step of reducinga value of the color difference data, determined to be within the colorrange in said determination step.
 5. The image processing methodaccording to claim 4, further comprising a step of removing a highfrequency component from the color difference data whose value isreduced in said reduction step.
 6. The image processing method accordingto claim 4, wherein the color range of the pseudo color componentincludes a color area from red to yellow, and a color area from blue tocyan.